Fine-grained lineage for safer notebook interactions

نویسندگان

چکیده

Computational notebooks have emerged as the platform of choice for data science and analytical workflows, enabling rapid iteration exploration. By keeping intermediate program state in memory segmenting units execution into so-called "cells", allow users to enjoy particularly tight feedback. However, cells are added, removed, reordered, rerun, this hidden accumulates, making behavior difficult reason about, leading errors lack reproducibility. We present nbsafety, a custom Jupyter kernel that uses runtime tracing static analysis automatically manage lineage associated with cell global notebook state. nbsafety detects prevents make during unaided interactions, all while preserving flexibility existing semantics. evaluate nbsafety's ability prevent erroneous interactions by replaying analyzing 666 real sessions. Of these, identified 117 sessions potential safety errors, remaining 549 sessions, resolving issues were more than 7X likely be selected re-execution compared random baseline, even though not using therefore influenced its suggestions.

برای دانلود باید عضویت طلایی داشته باشید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Smoke: Fine-grained Lineage at Interactive Speed

Data lineage describes the relationship between individual input and output data items of a workflow, and has served as an integral ingredient for both traditional (e.g., debugging, auditing, data integration, and security) and emergent (e.g., interactive visualizations, iterative analytics, explanations, and cleaning) applications. The core, long-standing problem that lineage systems need to a...

متن کامل

Supporting Fine-grained Data Lineage in a Database Visualization Environment

The lineage of a datum records its processing history. Because such information can be used to trace the source of anomalies and errors in processed data sets, it is valuable to users for a variety of applications including investigation of anomalies and debugging. Traditional data lineage approaches rely on metadata. However, metadata does not scale well to fine-grained lineage, especially in ...

متن کامل

Ultra-Fine Grained Dual-Phase Steels

This paper provides an overview on obtaining low-carbon ultra-fine grained dual-phase steels through rapid intercritical annealing of cold-rolled sheet as improved materials for automotive applications. A laboratory processing route was designed that involves cold-rolling of a tempered martensite structure followed by a second tempering step to produce a fine grained aggregate of ferrite and ca...

متن کامل

Spatio-Temporal Detection of Fine-Grained Dyadic Human Interactions

We introduce a novel spatio-temporal deformable part model for offline detection of fine-grained interactions in video. One novelty of the model is that part detectors model the interacting individuals in a single graph that can contain different combinations of feature descriptors. This allows us to use both body pose and movement to model the coordination between two people in space and time....

متن کامل

Attention for Fine-Grained Categorization

This paper presents experiments extending the work of Ba et al. (2014) on recurrent neural models for attention into less constrained visual environments, beginning with fine-grained categorization on the Stanford Dogs data set. In this work we use an RNN of the same structure but substitute a more powerful visual network and perform large-scale pre-training of the visual network outside of the...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: Proceedings of the VLDB Endowment

سال: 2021

ISSN: ['2150-8097']

DOI: https://doi.org/10.14778/3447689.3447712